• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The invention relates to a device (10) for predicting a level of vigilance of a conductor comprising a camera (11) for acquiring images of said conductor, an image processing module (13.1) for characterizing, from acquired images, physiological data of said driver, storage means of electroencephalographic and physiological data evolution profiles of a set of standard conductors acquired synchronously during driving tests during a phase of calibration, and a detection module (13.2) adapted to compare the evolution of the physiological data of said driver to the stored physiological data evolution profiles to determine at least one profile satisfying similarity criteria and to make a statistical prediction of the evolution of the level of vigilance of said driver from the analysis of the stored electroencephalographic data profile corresponding to the pr ofil physiological data evolution having satisfied the similarity criteria.
    公开号:FR3033303A1
    申请号:FR1551776
    申请日:2015-03-03
    公开日:2016-09-09
    发明作者:Jean-Philippe Alexander；Michele Moessinger；Ralph Bisping
    申请人:Renault SAS；
    IPC主号:
    专利说明:

    [0001] Apparatus and method for predicting a level of alertness in a driver of a motor vehicle. The invention relates to a device and a method for predicting a level of vigilance in a driver of a motor vehicle. An application of the method of the invention aims in particular to avoid hypovigilance in the driver. Hypovigilance typically corresponds to the transition between waking and sleeping, during which the body sees its powers of observation and analysis greatly reduced. This drop in the level of vigilance caused by the involuntary transition from waking to sleep is responsible for many accidents on the road. It therefore appears essential to be able to detect and prevent this accident-causing condition of the driver in order to improve his safety. Thus, to maintain a level of alertness and alertness consistent with the driving activity, drivers must take into account their driving time and take regular breaks, typically breaks every two hours are recommended. However, this requirement often frustrates the goal of making the journey as quickly as possible. Embedded systems are known to alert the driver when the vehicle's driving time exceeds two hours, but these warning systems are in themselves insufficient and, in particular, do not detect the real level of vigilance of the driver nor to monitor in real time the evolution of the level of vigilance of the driver. Also known are systems for detecting hypovigilance that use information recorded by electroencephalography, the analysis of brain activity (electroencephalogram or EEG) being recognized as being able to provide an indicator particularly representative of the state of alertness of a human being. However, if this technique makes it possible to evaluate the level of vigilance of a person with a great precision, the characterization of the EEG needs to be carried out using electrodes placed on the scalp. This method seems therefore little ergonomic or even to a certain extent dangerous if it is applied to the driver in action. No. 8,009,051 discloses a motor vehicle warning device capable of determining a driver's level of vigilance by generating driver stimulation and detecting the driver's response to the stimulation generated from a driver's response. analysis of a video signal of the driver provided by a camera embedded in the vehicle. Using different image processing and diagnostic techniques, the alerting device is designed to detect a driver's response to stimulation, for example based on a predetermined movement performed by the driver such as blinking, head or mouth movements. However, the evolution of the level of vigilance of a driver differs from one individual to another. Also, in order to be able to adapt this system to a set of drivers, the driver must interact with the system by confirming or invalidating the level of vigilance estimates made by the system, which is a source of distraction for the driver and therefore potentially a source of accident. The use of on-board cameras has also enabled the development of vigilance-level detection systems based on visual cues of the decline in alertness such as blinking, yawning or the direction of the driver's gaze. However, no system has proved sufficiently effective at the present time, in particular because hypovigilance manifests itself differently among the different individuals, which makes it difficult to define a common frame of reference for the population. detection of this phenomenon. Also, there is a need for a method of predicting a level of alertness in a driver, capable of providing good performance on a set of drivers. To this end, the invention relates to a method of predicting a level of vigilance of a driver, comprising: a calibration step, during which one acquires simultaneously, during driving tests: 3033303 3 electroencephalographic data evolution profiles of a set of typical conductors between a test start time when they are in a normal waking state and a test end time when they are in a state of hypovigilance; Physiological data evolution profiles associated with typical conductors by means of a camera; and a step of monitoring in real time the level of vigilance of the driver, during which, continuously: one acquires the evolution of physiological data of the driver by means of a camera; comparing the evolution of the physiological data of the driver to the different physiological data evolution profiles acquired during the calibration step, so as to determine among these profiles, at least one profile satisfying similarity criteria; A statistical prediction of the evolution of the level of vigilance of the driver is made from the analysis of the profile of electroencephalographic data acquired during the calibration step corresponding to the evolution profile of physiological data having satisfied the criteria of resemblance.
    [0002] The advantage of this method, in addition to a good rate of detection of hypovigilance states, is that it does not need to be adapted or trained. It is advantageously found that this method does not require learning on the driver. Advantageously, the statistical prediction of the evolution of the level of vigilance comprises the calculation of a probable score of vigilance of the driver able to define his level of vigilance on a continuous scale bounded between a value representative of a state of normal awakening. or very awake and a representative value of a state of hypovigilance or falling asleep. Preferably, the method may comprise a step of generating an alert signal in the event of a sudden change in the probable driver vigilance score.
    [0003] Advantageously, the method comprises a step of providing a high definition camera arranged to film at least the driver's face in a driving situation and a plurality of infrared LEDs intended to illuminate the field of the camera.
    [0004] Preferably, the method comprises a step of processing the images acquired by the camera comprising a real-time calculation of the physiological data associated with the driver able to characterize at least the blinks of the eyes and / or the yawning and / or the position and the movement of the driver's head.
    [0005] According to one embodiment, the real-time calculation of the physiological data associated with the driver comprises the determination of the parameters of frequency, speed, acceleration, and duration of opening and closing of the eyelids of the driver, the parameters of frequency of yawning and duration of opening and closing of the mouth during yawning, as well as position and movement parameters of the driver's head along three axes X, Y and Z. Advantageously, the method may comprise a step of customizing the statistical prediction comprising the transmission at regular intervals to the driver of a request for declaration of his level of vigilance estimated by him and the receipt of the level of vigilance estimated as declared by the driver. The invention also relates to a device for predicting a level of alertness of a driver, characterized in that it comprises a camera capable of acquiring images of the driver, an image processing module 25 able to characterize, from acquired images, physiological data of the driver, means for storing a database storing profiles of evolution of electroencephalographic data and of physiological data acquired synchronously during driving tests, during a phase calibration, 30 of a set of typical conductors put in a driving situation between a test start time where they are in a normal waking state and a test end time when they are in a state of hypovigilance, and a detection module 3033303 adapted to compare the evolution of the physiological data of the driver to the physiological data evolution profiles stored in the data base d. in such a way as to determine, among these profiles, at least one profile satisfying similarity criteria and to make a statistical prediction of the evolution of the level of vigilance of the driver from the analysis of the electroencephalographic data profile stored in the database corresponding to the evolution profile of physiological data having satisfied the similarity criteria. Advantageously, the device comprises a communication interface adapted to communicate with means of restitution of the statistical prediction of the evolution of the level of vigilance of the test conductor. Preferably, the rendering means comprise a human-machine interface arranged at a mobile terminal.
    [0006] Other features and advantages of the invention will emerge on reading the following description of a particular embodiment of the invention, given by way of non-limiting indication, with reference to the accompanying single figure illustrating schematically a device for detecting the level of vigilance of a driver according to the invention.
    [0007] The single figure schematically illustrates a device 10 for predicting the state of alertness of a driver, intended to be arranged in a vehicle. According to the configuration illustrated in the figure, the prediction device 10 comprises a camera 11, intended to be mounted in the vehicle, facing the driver, and intended to acquire video images of the driver of the vehicle, in particular video images of the vehicle. the part of the driver located above the bust of the driver, including in particular his head. The camera 11 used is preferably a high definition camera, continuously filming a black and white image. Preferably, the field of the camera 11 is illuminated by several infra-red LED light sources 12, 30 synchronized with the high definition camera and disposed on either side of the camera 11. These LEDs thus allow the camera to be able to film quality images by eliminating the reflections and large variations of contrast because of the large variations of the light environment in driving situation on the road. The device 10 comprises a processing unit 13, comprising an image processing module 13.1 designed to process the images acquired by the camera 11 by means of an image processing algorithm. This algorithm is designed to detect in real time in the acquired images of the physiological data of the driver, which concern the visual behavior of the driver and suitable to preferentially characterize at least the blinks of the eyes and / or the yawning and / or the position and the movement 10 of the test driver's head. In particular, according to one embodiment, the image processing algorithm 13.1 is designed to calculate in real time parameters of frequency, speed, acceleration, and duration of opening and closing of the eyelids of the driver, yawning frequency parameters and duration of opening and closing of the mouth during yawning, as well as position and movement parameters of the driver's head along the three axes X, Y and Z. A characterization of these different parameters are possible from the analysis of the video signal of the driver's face provided by the high-definition camera. We will not detail here the various known techniques of detection of faces or eyes used for the calculation of said parameters, but the skilled person can usefully refer to the existing literature on this subject. The characterization of these parameters thus makes it possible to carry out a real-time monitoring of the evolution of the physiological data of the driver that reveals his visual behavior when driving the vehicle. These parameters thus calculated by the image processing algorithm 13.1 are provided in real time to a detection module 13.2 designed to determine a level of vigilance of the driver by means of a statistical detection algorithm.
    [0008] This detection of the level of vigilance of the driver is carried out from a driving database comprising synchronized video and electroencephalographic recordings of a set, for example several hundreds, of standard or so-called conductors. calibration conductors. More precisely, during a calibration phase, a data acquisition campaign containing synchronized electroencephalographic and video recordings of this set of standard conductors is carried out during driving tests. this set of typical drivers put in driving situation between a moment of trial start where they are in a state of normal awakening and a moment of end of test where they are in a state of hypovigilance. Thus, during these driving tests in waking state evolving towards hypovigilance conditions, all the typical conductors 10 carry electrodes placed on their scalp, so as to acquire evolution profiles of electroencephalographic data. of these drivers. The typical drivers are furthermore filmed, for example by means of a high definition camera, during these driving tests, so as to simultaneously acquire physiological data evolution profiles associated with the typical conductors, which relate to the behavior. These types of conductors are visual and can be characterized according to the embodiment described above with reference to the image processing module 13.1 by parameters of frequency, speed, acceleration, and duration of opening and closing of eyelids 20 of the driver, yawning frequency parameters and duration of opening and closing of the mouth during yawning, as well as by parameters of position and movement of the driver's head along the three axes X, Y and Z. Thus, each record of the driving database 25 includes, for each typical driver having been put in driving situation between a start of driving in normal waking state and an end of driving time in a state of hypovigilance, a profile of evolution of electroencephalographic data and a video sequence, both acquired in a synchronized manner, the video sequence allowing to characterize an evolution profile of the visual behavior of the concerned type conductor according to the physiological data parameters previously stated.
    [0009] The detection module 13.2 then uses a statistical prediction algorithm constructed from the joint analysis of the physiological data and electroencephalographic data acquired during the driving tests and stored in the driving database. By comparing the evolution profiles of the physiological data with the evolution profiles of the electroencephalographic data, the statistical prediction algorithm is able to determine several models of statistical prediction of a level of vigilance as a function of the evolution of the physiological data acquired during the driving tests on the 10 typical conductors. More specifically, during a driver monitoring phase, during which the visual behavior of this conductor is filmed by the camera 11, the statistical detection module 13.2 is designed to compare in real time the evolution of the parameters of the driver. physiological data relating to the visual behavior of this conductor collected by means of the image processing module 13.1 as explained above, with the different physiological data evolution profiles previously recorded in the driving database during the calibration phase of the device, so as to determine among these registered profiles, the one or those satisfying similarity criteria. The module 13.2 then applies a statistical prediction model to make a statistical prediction of the level of vigilance of the driver from the analysis of the profiles of electroencephalographic data stored in the database corresponding to the evolution profiles of the physiological data. having satisfied the similarity criteria. The statistical prediction algorithm is designed to calculate a probable driver vigilance score, which enables the driver's level of alertness to be defined on a continuous scale bounded by a value representative of a normal or very awake state of awakening ( hypervigilance) and a representative value of a state of hypovigilance or falling asleep. For example, the probable driver vigilance score is given on a continuous scale from 0, corresponding to a state of hypovigilance, to 100, corresponding to a state of hypervigilance.
    [0010] The processing unit 13 of the detection device 10 also comprises an alert detection module 13.3 based on the probable driver vigilance score as established by the statistical detection module 13.2, making it possible to generate a signal alert to the driver in case of 5 abrupt evolution of the probable score of vigilance of the driver, in particular when this score decreases abruptly. The prediction device 10 comprises a wireless communication interface 14, in particular of the Bluetooth or WiFi or wired type, capable of communicating with a mobile terminal 15, in particular of the smartphone or tablet type, on which a software application dedicated to the reading and / or the management of the level of vigilance. The mobile terminal 15 includes a human-machine interface such as for example a touch screen 16 for the return of the probable score of vigilance to the driver. It also comprises a communication module enabling it to exchange data with the communication interface 14 of the prediction device 10. Thus, the detection device is able to send the probable score of vigilance of the driver who has been estimated to mobile terminal 15 via the communication interface 14 and the mobile terminal 15 is able to control the display of this score on the screen 16 of the mobile terminal 15 via the dedicated software application. Thus, the human-machine interface of the mobile terminal enables the driver to be informed in real time of his level of vigilance estimated by the prediction device 10. The human-machine interface of the mobile terminal can also be used to question the driver on his own feeling about his level of alertness, regularly, but also at the estimated level of vigilance crossings, for example among the following levels: very awake, normal, tired, very tired. The driver can thus be led, via the human-machine interface, to confirm or otherwise invalidate these level of vigilance crossings, so as to improve the performance of the statistical prediction algorithm as regards the personalized detection of the level of alertness. vigilance. Thus, with time and use, the prediction algorithm evolves from a global algorithm based on a multi-profile analysis of a set of typical drivers to a custom algorithm. 5 10
    权利要求:
    Claims (10)
    [0001]
    REVENDICATIONS1. A method for predicting a level of vigilance of a driver, characterized in that it comprises: a calibration step, during which one acquires simultaneously during driving tests: data evolution profiles electroencephalographic signals of a set of typical conductors between a trial start time when they are in a normal waking state and a test end time when they are in a state of hypovigilance; physiological data evolution profiles associated with the typical conductors by means of a camera; and a step of monitoring in real time the level of vigilance of the driver, during which, continuously: one acquires the evolution of physiological data of the driver by means of a camera (11); comparing the evolution of the physiological data of the driver to the different physiological data evolution profiles acquired during the calibration step, so as to determine among these profiles, at least one profile satisfying similarity criteria; a statistical prediction of the evolution of the level of vigilance of the driver is made from the analysis of the profile of electroencephalographic data acquired during the calibration step corresponding to the evolution profile of physiological data having satisfied the resemblance criteria .
    [0002]
    2. Method according to claim 1, characterized in that the statistical prediction of the evolution of the level of vigilance includes the calculation of a probable score of vigilance of the driver able to define his level of vigilance on a continuous scale bounded between a value representative of a waking state normal or very awake and a representative value of a state of hypovigilance or sleep. 3033303 12
    [0003]
    3. Method according to claim 2, characterized in that it comprises a step of generating an alert signal in case of sudden change in the probable score of vigilance of the driver.
    [0004]
    4. Method according to any one of the preceding claims, characterized in that it comprises a step of providing a high-definition camera (11) arranged to film at least the driver's face in a driving situation and a a plurality of infrared LEDs (12) for illuminating the field of the camera.
    [0005]
    5. Method according to any one of the preceding claims, characterized in that it comprises a step of processing the images acquired by the camera (11) comprising a real-time calculation of the physiological data associated with the driver capable of characterizing at least the blinking and / or yawning and / or position and movement of the test driver's head. 15
    [0006]
    6. Method according to claim 5, characterized in that the real-time calculation of the physiological data associated with the conductor comprises the determination of the parameters of frequency, speed, acceleration, and duration of opening and closing of the eyelids. conductive, yawning frequency parameters and duration of opening and closing of the mouth during yawning, as well as position and movement parameters of the driver's head along three axes X, Y and Z.
    [0007]
    7. Method according to any one of the preceding claims, characterized in that it comprises a step of personalizing the statistical prediction comprising the transmission at regular intervals to the driver of a request for declaration of his level of vigilance estimated by him and the receipt of the level of vigilance estimated as declared by the driver.
    [0008]
    8. Device for predicting (10) a vigilance level of a driver, characterized in that it comprises a camera (11) capable of acquiring images of the driver, a clean image processing module (13.1). characterizing, on the basis of the acquired images, physiological data of the driver, means for storing a database storing electroencephalographic data evolution profiles and physiological data acquired in a synchronized manner during test runs. conducting, during a calibration phase, a set of standard conductors placed in a driving situation between a test start time when they are in a normal waking state and a test end time where they are in a state of hypovigilance, and a detection module (13.2) adapted to compare the evolution of the physiological data of the driver to the evolution profiles of physiological data stored in the database of among these profiles, at least one profile satisfying similarity criteria is to be determined and a statistical prediction of the evolution of the level of vigilance of the driver based on the analysis of the electroencephalographic data profile stored in the base data corresponding to the evolution profile of physiological data having satisfied the resemblance criteria.
    [0009]
    9. Detection device according to claim 8, characterized in that it comprises a communication interface (14) adapted to communicate with means for restitution (15) of the statistical prediction of the evolution of the level of vigilance of the driver. 20
    [0010]
    10. Device according to claim 9, characterized in that the retrieval means comprise a man-machine interface (16) arranged at a mobile terminal. 25
    类似技术:
    公开号 | 公开日 | 专利标题
    EP3265334B1|2019-05-01|Device and method for predicting a vigilance level of a driver of a motor vehicle
    US9392956B2|2016-07-19|Dry sensor EEG/EMG and motion sensing system for seizure detection and monitoring
    EP2401963B1|2016-09-14|Driving attention amount determination device, method, and computer program
    FR3048544A1|2017-09-08|DEVICE AND METHOD FOR MONITORING A DRIVER OF A MOTOR VEHICLE
    JPWO2019188398A1|2021-05-13|Information processing equipment, mobile devices, and methods, and programs
    Karchani et al.2015|The Steps of proposed drowsiness detection system design based on image processing in simulator driving
    Ahlstrom et al.2012|Measuring driver impairments: sleepiness, distraction, and workload
    FR3048543A1|2017-09-08|DEVICE AND METHOD FOR MONITORING A CONDUCTOR OF A TRANSPORT VEHICLE
    Huang et al.2019|The impact of mind wandering on signal detection, semi-autonomous driving performance, and physiological responses
    AU2021100643A4|2021-04-15|Ai-based technology configured to enable physiological event prediction based on blepharometric data
    AU2021100637A4|2021-04-15|Blepharometric monitoring system for a vehicle which provides user-customised analysis
    AU2021100641A4|2021-04-15|Extended period blepharometric monitoring across multiple data collection platforms
    AU2020102426A4|2020-11-05|Collection of blepharometric data via a camera system
    Mabry et al.2019|Commercial motor vehicle operator fatigue detection technology catalog and review
    US20210378568A1|2021-12-09|Devices and processing systems configured to enable physiological event prediction based on blepharometric data analysis
    Alam et al.2021|Active Vision-Based Attention Monitoring System for Non-Distracted Driving
    AU2021105935A4|2021-12-02|System for determining physiological condition of driver in autonomous driving and alarming the driver using machine learning model
    US20190149777A1|2019-05-16|System for recording a scene based on scene content
    IL284881D0|2021-08-31|Detection of cognitive state of a driver
    US20210287697A1|2021-09-16|Techniques for separating driving emotion from media induced emotion in a driver monitoring system
    Vijay Kumar et al.2018|Automatic driver and vehicle safety monitoring system
    Cruz2016|Driver monitoring system based on eye tracking
    CN110826369A|2020-02-21|Driver attention detection method and system during driving
    FR3057517A1|2018-04-20|DEVICE FOR PREVENTING DANGEROUS SITUATIONS FOR A CONDUCTOR OF A TRANSPORT VEHICLE AND ASSOCIATED METHOD
    Jibran et al.2012|Securing Human Life By Fatigue Detection During Driving
    同族专利:
    公开号 | 公开日
    CN107428245B|2020-07-07|
    EP3265334A1|2018-01-10|
    JP6839090B2|2021-03-03|
    JP2018518205A|2018-07-12|
    CN107428245A|2017-12-01|
    WO2016139402A1|2016-09-09|
    FR3033303B1|2017-02-24|
    EP3265334B1|2019-05-01|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    WO2000044580A1|1999-01-27|2000-08-03|Compumedics Sleep Pty. Ltd.|Vigilance monitoring system|
    US7027621B1|2001-03-15|2006-04-11|Mikos, Ltd.|Method and apparatus for operator condition monitoring and assessment|
    DE102010007241A1|2010-02-09|2010-09-23|Daimler Ag|Driving route section determining method for driver assistance system of vehicle, involves recording electroencephalogram of test driver when driving on driving route, and determining route sections impairing attention of driver of vehicle|
    GB2491984A|2011-06-17|2012-12-19|Boeing Co|A display management system affected by the mental state of an operator and suited for use in aircraft cockpits|
    JP5056067B2|2007-02-26|2012-10-24|株式会社デンソー|Dozing alarm device|
    US8207859B2|2008-04-28|2012-06-26|Medtronic Minimed, Inc.|Automobile physiological monitoring system and method for using the same|
    CN101375796B|2008-09-18|2010-06-02|浙江工业大学|Real-time detection system of fatigue driving|
    CN101690666B|2009-10-13|2011-05-04|北京工业大学|Driving working load calculation method of automobile driver|
    US9405982B2|2013-01-18|2016-08-02|GM Global Technology Operations LLC|Driver gaze detection system|
    KR101526672B1|2013-07-24|2015-06-05|현대자동차주식회사|Apparatus and method for determining drowsy state|CN107137079B|2017-06-28|2020-12-08|京东方科技集团股份有限公司|Method for controlling equipment based on brain signals, control equipment and human-computer interaction system thereof|
    CN109803583A|2017-08-10|2019-05-24|北京市商汤科技开发有限公司|Driver monitoring method, apparatus and electronic equipment|
    FR3078518B1|2018-03-05|2020-02-07|Psa Automobiles Sa|ASSISTANCE DEVICE FOR A VEHICLE DRIVER BY DETERMINING A STATE OF VIGILANCE WITH ADAPTED DECISION RULES|
    CN112455461A|2019-09-06|2021-03-09|华为技术有限公司|Human-vehicle interaction method for automatically driving vehicle and automatically driving system|
    法律状态:
    2016-03-21| PLFP| Fee payment|Year of fee payment: 2 |
    2016-09-09| PLSC| Publication of the preliminary search report|Effective date: 20160909 |
    2017-03-22| PLFP| Fee payment|Year of fee payment: 3 |
    2018-03-23| PLFP| Fee payment|Year of fee payment: 4 |
    2020-03-19| PLFP| Fee payment|Year of fee payment: 6 |
    2021-12-10| ST| Notification of lapse|Effective date: 20211105 |
    优先权:
    申请号 | 申请日 | 专利标题
    FR1551776A|FR3033303B1|2015-03-03|2015-03-03|DEVICE AND METHOD FOR PREDICTING A LEVEL OF VIGILANCE IN A DRIVER OF A MOTOR VEHICLE.|FR1551776A| FR3033303B1|2015-03-03|2015-03-03|DEVICE AND METHOD FOR PREDICTING A LEVEL OF VIGILANCE IN A DRIVER OF A MOTOR VEHICLE.|
    JP2017545556A| JP6839090B2|2015-03-03|2016-02-19|Devices and methods for predicting the arousal level of a motor vehicle driver|
    CN201680018126.9A| CN107428245B|2015-03-03|2016-02-19|Device and method for predicting the level of alertness of a driver of a motor vehicle|
    EP16714471.6A| EP3265334B1|2015-03-03|2016-02-19|Device and method for predicting a vigilance level of a driver of a motor vehicle|
    PCT/FR2016/050396| WO2016139402A1|2015-03-03|2016-02-19|Device and method for predicting a vigilance level of a driver of a motor vehicle|
    [返回顶部]